Compare HDR versus Burst using cpBurstCamera
Experiment with various HDR (multiple frames at different exposures) and other multi-image "burst" algorithms for capturing scenes with a variety of dynamic ranges, as well as object and camera motion.
Create a pbrt scene we can use
Using pbrt lets us get the highest-fidelity rendering
pbrtLensFile = false; % Doesn't work on Windows currently
sceneChoice = "CornellBoxReference";
apertureDiameter = 6;% in mm
% for legacy reasons, scenes don't have a uniform naming scheme,
% so we map the user's choice to the appropriate options
if strcmp(sceneChoice, "Cornell Box with Bunny")
scenePath = 'Cornell_BoxBunnyChart';
sceneName = 'cornell box bunny chart';
elseif strcmp(sceneChoice, "CornellBoxReference")
scenePath = "CornellBoxReference";
sceneName = "CornellBoxReference";
pbrtCPScene = cpScene('pbrt', 'scenePath', scenePath, 'sceneName', sceneName, ...
'resolution', [filmResolution filmResolution], ...
'numRays', numRays, 'sceneLuminance', sceneLuminance, ...
'lensFile','dgauss.22deg.6.0mm.json',...
'apertureDiameter', apertureDiameter);
pbrtCPScene = cpScene('pbrt', 'scenePath', scenePath, 'sceneName', sceneName, ...
'resolution', [filmResolution filmResolution], ...
'numRays', numRays, 'sceneLuminance', sceneLuminance, 'verbose', 0);
end
Read 1 materials.
Read 1 textures.
***Scene parsed.
Create our sample camera
Just one module with a pre-made sensor and default optics.
% some timing code, just to see how fast we run...
setpref('ISET', 'benchmarkstart', cputime);
setpref('ISET', 'tStart', tic);
% cpBurstCamera is a sub-class of cpCamera that implements simple HDR and Burst
ourCamera = cpBurstCamera();
% We'll use a pre-defined sensor for our Camera Module, and let it use
% default optics for now. We can then assign the module to our camera:
sensor = sensorFromFile('ar0132atSensorRGB');
% Cameras can eventually have more than one module (lens + sensor)
% but for now, we just create one using our sensor
ourCamera.cmodules(1) = cpCModule('sensor', sensor); % default
Let's Look at the Camera we've Created
cpCameraWindow is actually a mini-application, that lets you see the properties of your camera, and the scene you've chosen. You can use it to take photos in whatever modes your camera supports. It also allows to to set Advanced Processing, which tells cameras that support it to do additional processing like image registration and tonemapping, as needed, in their ISP, instead of the default combination of frames in the sensor.
cpCameraWindow(ourCamera, pbrtCPScene);
Take Pictures using Auto, HDR, and Burst Intents
For the simplest case we ask our camera to take some pictures using pre-defined intents. By deault our camera assumes the 'Auto' intent. We also demonstrate 'Burst', which will capture more frames if the camera type we are using supports it. 'HDR' also fires a burst, but bracketed. Our default camera sums the frames in Burst mode, and chooses the highest non-saturated value at each pixel for HDR mode.
autoImage = ourCamera.TakePicture(pbrtCPScene, 'Auto',...
'imageName','Auto Mode no motion', 'insensorIP',false);
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp74186a73_9321_4484_a9d3_6a05a7db0d05\CornellBoxReference.pbrt
Elapsed time is 0.256886 seconds.
*** Rendering time for CornellBoxReference: 73.4 sec ***
*** Rendering time for CornellBoxReference_depth: 22.7 sec ***
Docker container vistalab/pbrt-v3-spectral:latest
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\CornellBoxReference_depth.pbrt
Docker command
docker run -ti --rm -w /CornellBoxReference -v C:/iset/iset3d/local/CornellBoxReference:/CornellBoxReference vistalab/pbrt-v3-spectral:latest pbrt --outfile renderings/CornellBoxReference_depth.dat CornellBoxReference_depth.pbrt
*** Rendering time for CornellBoxReference_depth: 22.2 sec ***
Reading image h=512 x w=512 x 31 spectral planes.
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpf2c844f0_3c3d_4d23_9137_8e8e025316c6\CornellBoxReference.pbrt
Elapsed time is 0.202212 seconds.
*** Rendering time for CornellBoxReference: 76.2 sec ***
*** Rendering time for CornellBoxReference_depth: 23.2 sec ***
hdrSensorImage = ourCamera.TakePicture(pbrtCPScene, 'HDR',...
'numHDRFrames', 3,'imageName','HDR in sensor no motion', ...
'insensorIP', true);
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpcacb3bed_3365_4d3e_a719_7321e4d0637d\CornellBoxReference.pbrt
Elapsed time is 0.211676 seconds.
*** Rendering time for CornellBoxReference: 71.3 sec ***
*** Rendering time for CornellBoxReference_depth: 22.9 sec ***
Docker container vistalab/pbrt-v3-spectral:latest
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\CornellBoxReference_depth.pbrt
Docker command
docker run -ti --rm -w /CornellBoxReference -v C:/iset/iset3d/local/CornellBoxReference:/CornellBoxReference vistalab/pbrt-v3-spectral:latest pbrt --outfile renderings/CornellBoxReference_depth.dat CornellBoxReference_depth.pbrt
*** Rendering time for CornellBoxReference_depth: 21.1 sec ***
Reading image h=512 x w=512 x 31 spectral planes.
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp320afa3a_3e96_4f87_87e7_f6cb46bd55d9\CornellBoxReference.pbrt
Elapsed time is 0.214805 seconds.
*** Rendering time for CornellBoxReference: 70.2 sec ***
*** Rendering time for CornellBoxReference_depth: 24.8 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpfc64ae64_4979_4cb5_a517_17109ae9e89c\CornellBoxReference.pbrt
Elapsed time is 0.265101 seconds.
*** Rendering time for CornellBoxReference: 74.3 sec ***
*** Rendering time for CornellBoxReference_depth: 21.2 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpdac0b3bf_c408_4282_ab65_18f79c0fe738\CornellBoxReference.pbrt
Elapsed time is 0.243284 seconds.
*** Rendering time for CornellBoxReference: 76.3 sec ***
*** Rendering time for CornellBoxReference_depth: 24.0 sec ***
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.48 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.37 (6 of 8)
Depth 0.11 Defocus -8.95 (7 of 8)
Depth 0.04 Defocus -26.84 (8 of 8)
Depth 0.56 Defocus 0.77 (1 of 8)
Depth 0.48 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.12 (3 of 8)
Depth 0.34 Defocus -0.42 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.81 (6 of 8)
Depth 0.11 Defocus -6.38 (7 of 8)
Depth 0.04 Defocus -24.28 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.48 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.27 (6 of 8)
Depth 0.11 Defocus -3.85 (7 of 8)
Depth 0.04 Defocus -21.74 (8 of 8)
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.48 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.37 (6 of 8)
Depth 0.11 Defocus -8.95 (7 of 8)
Depth 0.04 Defocus -26.84 (8 of 8)
Depth 0.56 Defocus 0.77 (1 of 8)
Depth 0.48 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.12 (3 of 8)
Depth 0.34 Defocus -0.42 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.81 (6 of 8)
Depth 0.11 Defocus -6.38 (7 of 8)
Depth 0.04 Defocus -24.28 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.48 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.27 (6 of 8)
Depth 0.11 Defocus -3.85 (7 of 8)
Depth 0.04 Defocus -21.74 (8 of 8)
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.48 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.37 (6 of 8)
Depth 0.11 Defocus -8.95 (7 of 8)
Depth 0.04 Defocus -26.84 (8 of 8)
Depth 0.56 Defocus 0.77 (1 of 8)
Depth 0.48 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.12 (3 of 8)
Depth 0.34 Defocus -0.42 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.81 (6 of 8)
Depth 0.11 Defocus -6.38 (7 of 8)
Depth 0.04 Defocus -24.28 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.48 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.27 (6 of 8)
Depth 0.11 Defocus -3.85 (7 of 8)
Depth 0.04 Defocus -21.74 (8 of 8)
burstSensorImage = ourCamera.TakePicture(pbrtCPScene, 'Burst',...
'numBurstFrames', 3, 'imageName','Burst in Sensor',...
'insensorIP',true);
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp4e52e99e_1190_421e_b726_a7ae89aa685f\CornellBoxReference.pbrt
Elapsed time is 0.221141 seconds.
*** Rendering time for CornellBoxReference: 79.3 sec ***
*** Rendering time for CornellBoxReference_depth: 25.5 sec ***
Docker container vistalab/pbrt-v3-spectral:latest
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\CornellBoxReference_depth.pbrt
Docker command
docker run -ti --rm -w /CornellBoxReference -v C:/iset/iset3d/local/CornellBoxReference:/CornellBoxReference vistalab/pbrt-v3-spectral:latest pbrt --outfile renderings/CornellBoxReference_depth.dat CornellBoxReference_depth.pbrt
*** Rendering time for CornellBoxReference_depth: 24.5 sec ***
Reading image h=512 x w=512 x 31 spectral planes.
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpb74dc39a_3892_4eaf_a415_9717ee8f61ff\CornellBoxReference.pbrt
Elapsed time is 0.253710 seconds.
*** Rendering time for CornellBoxReference: 77.2 sec ***
*** Rendering time for CornellBoxReference_depth: 24.4 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpd6447241_508a_4830_9a0b_9c121d8ad792\CornellBoxReference.pbrt
Elapsed time is 0.261052 seconds.
*** Rendering time for CornellBoxReference: 74.4 sec ***
*** Rendering time for CornellBoxReference_depth: 21.7 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp6bdecd66_fd04_4dc9_9b71_4b0bd217e4eb\CornellBoxReference.pbrt
Elapsed time is 0.274828 seconds.
*** Rendering time for CornellBoxReference: 73.3 sec ***
*** Rendering time for CornellBoxReference_depth: 23.0 sec ***
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.48 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.37 (6 of 8)
Depth 0.11 Defocus -8.95 (7 of 8)
Depth 0.04 Defocus -26.84 (8 of 8)
Depth 0.56 Defocus 0.77 (1 of 8)
Depth 0.48 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.12 (3 of 8)
Depth 0.34 Defocus -0.42 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.81 (6 of 8)
Depth 0.11 Defocus -6.38 (7 of 8)
Depth 0.04 Defocus -24.28 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.48 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.27 (6 of 8)
Depth 0.11 Defocus -3.85 (7 of 8)
Depth 0.04 Defocus -21.74 (8 of 8)
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.48 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.37 (6 of 8)
Depth 0.11 Defocus -8.95 (7 of 8)
Depth 0.04 Defocus -26.84 (8 of 8)
Depth 0.56 Defocus 0.77 (1 of 8)
Depth 0.48 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.12 (3 of 8)
Depth 0.34 Defocus -0.42 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.81 (6 of 8)
Depth 0.11 Defocus -6.38 (7 of 8)
Depth 0.04 Defocus -24.28 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.48 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.27 (6 of 8)
Depth 0.11 Defocus -3.85 (7 of 8)
Depth 0.04 Defocus -21.74 (8 of 8)
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.48 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.37 (6 of 8)
Depth 0.11 Defocus -8.95 (7 of 8)
Depth 0.04 Defocus -26.84 (8 of 8)
Depth 0.56 Defocus 0.77 (1 of 8)
Depth 0.48 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.12 (3 of 8)
Depth 0.34 Defocus -0.42 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.81 (6 of 8)
Depth 0.11 Defocus -6.38 (7 of 8)
Depth 0.04 Defocus -24.28 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.48 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.27 (6 of 8)
Depth 0.11 Defocus -3.85 (7 of 8)
Depth 0.04 Defocus -21.74 (8 of 8)
compareImages(hdrSensorImage, burstSensorImage, 'HDR vs Burst with no moion');
HDR vs Burst with no moion
Now we add a small amount of camera motion
Combining multiple frames into a single image can either be done at the sensor level, or a higher-level in the ISP. At the sensor level the default strategies implemented by ISETCam are to sum the individual frames for Burst mode, and select the image with the most signal at each pixel for HDR. Once there is camera motion, though, those algorithms create errors in registration.
So our BurstCamera allows us to turn on Enhanced Processing, which does HDR by rendering each frame and then using image registration and tone-mapping to create the result. Both are "off-the-shelf" algorithms, so one place for experimentation and innovation is improving upon them, perhaps by using some of the papers from Google and others. For Burst, only image registration is used. This simple form of image registration relies on a single geometric transformation of the entire image, so it does not help with object motion.
pbrtCPScene.cameraMotion = {{'unused', [.002, 0, 0], [0, 0, 0]}}; % first tuple is translation, second is rotation
hdrMotionImage = ourCamera.TakePicture(pbrtCPScene, 'HDR',...
'numHDRFrames', 3,'imageName','HDR in ISP with Motion', ...
'insensorIP', true);
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp394195d6_052b_43bf_b7df_8c88a78edfd3\CornellBoxReference.pbrt
Elapsed time is 0.228763 seconds.
*** Rendering time for CornellBoxReference: 75.8 sec ***
*** Rendering time for CornellBoxReference_depth: 21.9 sec ***
Docker container vistalab/pbrt-v3-spectral:latest
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\CornellBoxReference_depth.pbrt
Docker command
docker run -ti --rm -w /CornellBoxReference -v C:/iset/iset3d/local/CornellBoxReference:/CornellBoxReference vistalab/pbrt-v3-spectral:latest pbrt --outfile renderings/CornellBoxReference_depth.dat CornellBoxReference_depth.pbrt
*** Rendering time for CornellBoxReference_depth: 23.1 sec ***
Reading image h=512 x w=512 x 31 spectral planes.
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpe8ccc57f_ace0_4785_8f1d_6fceeca593f0\CornellBoxReference.pbrt
Elapsed time is 0.235368 seconds.
*** Rendering time for CornellBoxReference: 82.3 sec ***
*** Rendering time for CornellBoxReference_depth: 22.8 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp091a0ad3_aebc_4379_8b19_f339820f2d3e\CornellBoxReference.pbrt
Elapsed time is 0.244556 seconds.
*** Rendering time for CornellBoxReference: 74.9 sec ***
*** Rendering time for CornellBoxReference_depth: 21.7 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpecbfa5a3_7965_4c05_858a_52cc60feca46\CornellBoxReference.pbrt
Elapsed time is 0.243802 seconds.
*** Rendering time for CornellBoxReference: 77.2 sec ***
*** Rendering time for CornellBoxReference_depth: 23.7 sec ***
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.48 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.37 (6 of 8)
Depth 0.11 Defocus -8.95 (7 of 8)
Depth 0.04 Defocus -26.84 (8 of 8)
Depth 0.56 Defocus 0.77 (1 of 8)
Depth 0.48 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.12 (3 of 8)
Depth 0.34 Defocus -0.42 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.81 (6 of 8)
Depth 0.11 Defocus -6.38 (7 of 8)
Depth 0.04 Defocus -24.28 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.48 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.27 (6 of 8)
Depth 0.11 Defocus -3.85 (7 of 8)
Depth 0.04 Defocus -21.74 (8 of 8)
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.48 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.36 (6 of 8)
Depth 0.11 Defocus -8.94 (7 of 8)
Depth 0.04 Defocus -26.82 (8 of 8)
Depth 0.56 Defocus 0.78 (1 of 8)
Depth 0.48 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.12 (3 of 8)
Depth 0.34 Defocus -0.42 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.80 (6 of 8)
Depth 0.11 Defocus -6.38 (7 of 8)
Depth 0.04 Defocus -24.26 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.48 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.26 (6 of 8)
Depth 0.11 Defocus -3.84 (7 of 8)
Depth 0.04 Defocus -21.72 (8 of 8)
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.49 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.36 (6 of 8)
Depth 0.11 Defocus -8.93 (7 of 8)
Depth 0.04 Defocus -26.80 (8 of 8)
Depth 0.56 Defocus 0.78 (1 of 8)
Depth 0.49 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.13 (3 of 8)
Depth 0.34 Defocus -0.41 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.80 (6 of 8)
Depth 0.11 Defocus -6.37 (7 of 8)
Depth 0.04 Defocus -24.24 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.49 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.26 (6 of 8)
Depth 0.11 Defocus -3.83 (7 of 8)
Depth 0.04 Defocus -21.70 (8 of 8)
compareImages(hdrSensorImage, hdrMotionImage, 'In sensor HDR without and with motion');
In sensor HDR without and with motion
% Now try with firmware ISP ("Enhanced Processing" setting)
hdrISPMotionImage = ourCamera.TakePicture(pbrtCPScene, 'HDR',...
'numHDRFrames', 3,'imageName','HDR in ISP with Motion', ...
'insensorIP', false);
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp70bf216f_ae5e_4ca2_97df_ad13bfadc07e\CornellBoxReference.pbrt
Elapsed time is 0.223022 seconds.
*** Rendering time for CornellBoxReference: 75.0 sec ***
*** Rendering time for CornellBoxReference_depth: 21.7 sec ***
Docker container vistalab/pbrt-v3-spectral:latest
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\CornellBoxReference_depth.pbrt
Docker command
docker run -ti --rm -w /CornellBoxReference -v C:/iset/iset3d/local/CornellBoxReference:/CornellBoxReference vistalab/pbrt-v3-spectral:latest pbrt --outfile renderings/CornellBoxReference_depth.dat CornellBoxReference_depth.pbrt
*** Rendering time for CornellBoxReference_depth: 20.0 sec ***
Reading image h=512 x w=512 x 31 spectral planes.
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpd38ce3cf_75f6_4ef1_aae5_baec11adc7f6\CornellBoxReference.pbrt
Elapsed time is 0.238994 seconds.
*** Rendering time for CornellBoxReference: 72.4 sec ***
*** Rendering time for CornellBoxReference_depth: 23.3 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpd37c8d7a_5ccd_4cf7_8195_e65b1adb4e1a\CornellBoxReference.pbrt
Elapsed time is 0.265558 seconds.
*** Rendering time for CornellBoxReference: 73.4 sec ***
*** Rendering time for CornellBoxReference_depth: 22.2 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp513e6831_726e_4537_84f3_011e5874ad50\CornellBoxReference.pbrt
Elapsed time is 0.506689 seconds.
*** Rendering time for CornellBoxReference: 72.2 sec ***
*** Rendering time for CornellBoxReference_depth: 21.4 sec ***
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.49 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.44 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.36 (6 of 8)
Depth 0.11 Defocus -8.93 (7 of 8)
Depth 0.04 Defocus -26.80 (8 of 8)
Depth 0.56 Defocus 0.78 (1 of 8)
Depth 0.49 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.13 (3 of 8)
Depth 0.34 Defocus -0.41 (4 of 8)
Depth 0.26 Defocus -1.27 (5 of 8)
Depth 0.19 Defocus -2.80 (6 of 8)
Depth 0.11 Defocus -6.37 (7 of 8)
Depth 0.04 Defocus -24.24 (8 of 8)
Depth 0.56 Defocus 3.31 (1 of 8)
Depth 0.49 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.66 (3 of 8)
Depth 0.34 Defocus 2.12 (4 of 8)
Depth 0.26 Defocus 1.27 (5 of 8)
Depth 0.19 Defocus -0.26 (6 of 8)
Depth 0.11 Defocus -3.83 (7 of 8)
Depth 0.04 Defocus -21.70 (8 of 8)
Depth 0.56 Defocus -1.79 (1 of 8)
Depth 0.49 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.43 (3 of 8)
Depth 0.34 Defocus -2.98 (4 of 8)
Depth 0.26 Defocus -3.83 (5 of 8)
Depth 0.19 Defocus -5.36 (6 of 8)
Depth 0.11 Defocus -8.93 (7 of 8)
Depth 0.04 Defocus -26.78 (8 of 8)
Depth 0.56 Defocus 0.78 (1 of 8)
Depth 0.49 Defocus 0.50 (2 of 8)
Depth 0.41 Defocus 0.13 (3 of 8)
Depth 0.34 Defocus -0.41 (4 of 8)
Depth 0.26 Defocus -1.26 (5 of 8)
Depth 0.19 Defocus -2.79 (6 of 8)
Depth 0.11 Defocus -6.36 (7 of 8)
Depth 0.04 Defocus -24.21 (8 of 8)
Depth 0.56 Defocus 3.32 (1 of 8)
Depth 0.49 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.67 (3 of 8)
Depth 0.34 Defocus 2.13 (4 of 8)
Depth 0.26 Defocus 1.28 (5 of 8)
Depth 0.19 Defocus -0.25 (6 of 8)
Depth 0.11 Defocus -3.82 (7 of 8)
Depth 0.04 Defocus -21.68 (8 of 8)
Depth 0.56 Defocus -1.78 (1 of 8)
Depth 0.49 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.43 (3 of 8)
Depth 0.34 Defocus -2.97 (4 of 8)
Depth 0.26 Defocus -3.82 (5 of 8)
Depth 0.19 Defocus -5.35 (6 of 8)
Depth 0.11 Defocus -8.92 (7 of 8)
Depth 0.04 Defocus -26.75 (8 of 8)
Depth 0.56 Defocus 0.78 (1 of 8)
Depth 0.49 Defocus 0.51 (2 of 8)
Depth 0.41 Defocus 0.13 (3 of 8)
Depth 0.34 Defocus -0.41 (4 of 8)
Depth 0.26 Defocus -1.26 (5 of 8)
Depth 0.19 Defocus -2.79 (6 of 8)
Depth 0.11 Defocus -6.35 (7 of 8)
Depth 0.04 Defocus -24.19 (8 of 8)
Depth 0.56 Defocus 3.32 (1 of 8)
Depth 0.49 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.67 (3 of 8)
Depth 0.34 Defocus 2.13 (4 of 8)
Depth 0.26 Defocus 1.28 (5 of 8)
Depth 0.19 Defocus -0.25 (6 of 8)
Depth 0.11 Defocus -3.82 (7 of 8)
Depth 0.04 Defocus -21.65 (8 of 8)
compareImages(hdrISPMotionImage, hdrMotionImage, 'Tonemap vs. In sensor HDR in Camera Motion Case');
Tonemap vs. In sensor HDR in Camera Motion Case
burstMotionImage = ourCamera.TakePicture(pbrtCPScene, 'Burst',...
'numBurstFrames', 3, 'imageName','Burst in ISP with Motion',...
'insensorIP',false);
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp52156bc1_de3b_41c9_8e87_02d8d7480ba5\CornellBoxReference.pbrt
Elapsed time is 0.228538 seconds.
*** Rendering time for CornellBoxReference: 73.9 sec ***
*** Rendering time for CornellBoxReference_depth: 21.4 sec ***
Docker container vistalab/pbrt-v3-spectral:latest
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\CornellBoxReference_depth.pbrt
Docker command
docker run -ti --rm -w /CornellBoxReference -v C:/iset/iset3d/local/CornellBoxReference:/CornellBoxReference vistalab/pbrt-v3-spectral:latest pbrt --outfile renderings/CornellBoxReference_depth.dat CornellBoxReference_depth.pbrt
*** Rendering time for CornellBoxReference_depth: 22.6 sec ***
Reading image h=512 x w=512 x 31 spectral planes.
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tp9c0eca2a_ac5b_4473_ba7f_046ec07b36be\CornellBoxReference.pbrt
Elapsed time is 0.288769 seconds.
*** Rendering time for CornellBoxReference: 71.3 sec ***
*** Rendering time for CornellBoxReference_depth: 23.1 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpe1af0de3_0c8c_4758_9f85_cf048ca72028\CornellBoxReference.pbrt
Elapsed time is 0.269873 seconds.
*** Rendering time for CornellBoxReference: 71.2 sec ***
*** Rendering time for CornellBoxReference_depth: 23.2 sec ***
Overwriting PBRT file C:\iset\iset3d\local\CornellBoxReference\tpec15c54b_e375_4a73_8174_bb97eb29e4f3\CornellBoxReference.pbrt
Elapsed time is 0.250779 seconds.
*** Rendering time for CornellBoxReference: 72.2 sec ***
*** Rendering time for CornellBoxReference_depth: 23.8 sec ***
Depth 0.56 Defocus -1.78 (1 of 8)
Depth 0.49 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.43 (3 of 8)
Depth 0.34 Defocus -2.97 (4 of 8)
Depth 0.26 Defocus -3.82 (5 of 8)
Depth 0.19 Defocus -5.35 (6 of 8)
Depth 0.11 Defocus -8.92 (7 of 8)
Depth 0.04 Defocus -26.75 (8 of 8)
Depth 0.56 Defocus 0.78 (1 of 8)
Depth 0.49 Defocus 0.51 (2 of 8)
Depth 0.41 Defocus 0.13 (3 of 8)
Depth 0.34 Defocus -0.41 (4 of 8)
Depth 0.26 Defocus -1.26 (5 of 8)
Depth 0.19 Defocus -2.79 (6 of 8)
Depth 0.11 Defocus -6.35 (7 of 8)
Depth 0.04 Defocus -24.19 (8 of 8)
Depth 0.56 Defocus 3.32 (1 of 8)
Depth 0.49 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.67 (3 of 8)
Depth 0.34 Defocus 2.13 (4 of 8)
Depth 0.26 Defocus 1.28 (5 of 8)
Depth 0.19 Defocus -0.25 (6 of 8)
Depth 0.11 Defocus -3.82 (7 of 8)
Depth 0.04 Defocus -21.65 (8 of 8)
Depth 0.56 Defocus -1.78 (1 of 8)
Depth 0.49 Defocus -2.06 (2 of 8)
Depth 0.41 Defocus -2.43 (3 of 8)
Depth 0.34 Defocus -2.97 (4 of 8)
Depth 0.26 Defocus -3.82 (5 of 8)
Depth 0.19 Defocus -5.35 (6 of 8)
Depth 0.11 Defocus -8.91 (7 of 8)
Depth 0.04 Defocus -26.73 (8 of 8)
Depth 0.56 Defocus 0.78 (1 of 8)
Depth 0.49 Defocus 0.51 (2 of 8)
Depth 0.41 Defocus 0.13 (3 of 8)
Depth 0.34 Defocus -0.41 (4 of 8)
Depth 0.26 Defocus -1.26 (5 of 8)
Depth 0.19 Defocus -2.78 (6 of 8)
Depth 0.11 Defocus -6.35 (7 of 8)
Depth 0.04 Defocus -24.17 (8 of 8)
Depth 0.56 Defocus 3.32 (1 of 8)
Depth 0.49 Defocus 3.04 (2 of 8)
Depth 0.41 Defocus 2.67 (3 of 8)
Depth 0.34 Defocus 2.13 (4 of 8)
Depth 0.26 Defocus 1.28 (5 of 8)
Depth 0.19 Defocus -0.25 (6 of 8)
Depth 0.11 Defocus -3.81 (7 of 8)
Depth 0.04 Defocus -21.63 (8 of 8)
Depth 0.56 Defocus -1.78 (1 of 8)
Depth 0.49 Defocus -2.05 (2 of 8)
Depth 0.41 Defocus -2.43 (3 of 8)
Depth 0.34 Defocus -2.97 (4 of 8)
Depth 0.26 Defocus -3.82 (5 of 8)
Depth 0.19 Defocus -5.34 (6 of 8)
Depth 0.11 Defocus -8.90 (7 of 8)
Depth 0.04 Defocus -26.71 (8 of 8)
Depth 0.56 Defocus 0.78 (1 of 8)
Depth 0.49 Defocus 0.51 (2 of 8)
Depth 0.41 Defocus 0.14 (3 of 8)
Depth 0.34 Defocus -0.40 (4 of 8)
Depth 0.26 Defocus -1.25 (5 of 8)
Depth 0.19 Defocus -2.78 (6 of 8)
Depth 0.11 Defocus -6.34 (7 of 8)
Depth 0.04 Defocus -24.15 (8 of 8)
Depth 0.56 Defocus 3.32 (1 of 8)
Depth 0.49 Defocus 3.05 (2 of 8)
Depth 0.41 Defocus 2.67 (3 of 8)
Depth 0.34 Defocus 2.13 (4 of 8)
Depth 0.26 Defocus 1.29 (5 of 8)
Depth 0.19 Defocus -0.24 (6 of 8)
Depth 0.11 Defocus -3.80 (7 of 8)
Depth 0.04 Defocus -21.61 (8 of 8)
compareImages(hdrISPMotionImage, burstMotionImage,"HDR vs. Burst in ISP for Camera Motion Case");
HDR vs. Burst in ISP for Camera Motion Case
Print out some timing stats
Just for human consumption
tTotal = toc(getpref('ISET','tStart'));
beforeTime = getpref('ISET', 'benchmarkstart', 0);
disp(strcat("cpCompare ran on: ", glData.Vendor, " ", glData.Renderer, "with driver version: ", glData.Version));
cpCompare ran on: NVIDIA Corporation Quadro T2000/PCIe/SSE2with driver version: 4.6.0 NVIDIA 472.12
disp(strcat("cpCompare ran in: ", string(afterTime - beforeTime), " seconds of CPU time."));
cpCompare ran in: 2002.4375 seconds of CPU time.
disp(strcat("cpCompare ran in: ", string(tTotal), " total seconds."));
cpCompare ran in: 4465.158 total seconds.
This helper function (compareImages) allows us to compare results whether multi-image processing is done in sensor, or in the ISP (sensor returns an IP, ISP returns and RGB)
function compareImages(leftImage, rightImage, title)
leftImageRendered = imageShowImage(leftImage, [],[],0);
leftImageRendered = leftImage;
rightImageRendered = imageShowImage(rightImage,[],[],0);
rightImageRendered = rightImage;
% in a Live Script figure titles don't show until they are popped-out
imshowpair(leftImageRendered,rightImageRendered, 'montage');
text(.1,.1,title, 'Color','red','FontSize',28,'Units','normalized');
